Brandon/flux model loading #6739

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

brandonrising merged 113 commits into main from brandon/flux-model-loading

Aug 27, 2024

Contributor

brandonrising commented Aug 12, 2024 •

edited

Loading

Summary

Add FLUX support to the model manager.

QA Instructions

Install the FLUX model via the "Starter Models" tab:

Test fresh install and inference of all FLUX variants
- FLUX schnell
- FLUX schnell quantized
- FLUX dev
- FLUX dev quantized
Test quantized on 16GB GPU
Maybe: Test quantized on 8GB GPU
Test that the app still runs on CPU and MPS systems (main concern is pip install, or that we try to import bitsandbytes)

Merge Plan

Can be merged with proper approvals

github-actions bot added python invocations backend frontend labels

brandonrising commented

View reviewed changes

invokeai/backend/model_manager/probe.py Outdated Show resolved Hide resolved

brandonrising commented

View reviewed changes

invokeai/backend/quantization/fast_quantized_diffusion_model.py Outdated Show resolved Hide resolved

brandonrising commented

View reviewed changes

invokeai/backend/quantization/fast_quantized_transformers_model.py Outdated Show resolved Hide resolved

sammcj mentioned this pull request

[enhancement]: Please support FLUX.1 #6712

Closed

1 task

brandonrising force-pushed the brandon/flux-model-loading branch from 2e978ad to 0e14fc6 Compare

August 15, 2024 14:27

github-actions bot added Root python-deps services labels

brandonrising force-pushed the brandon/flux-model-loading branch from 043df07 to 00b63af Compare

August 19, 2024 17:13

RyanJDick changed the base branch from ryan/flux to main

August 20, 2024 18:10

github-actions bot added the python-tests label

RyanJDick reviewed

View reviewed changes

invokeai/app/invocations/fields.py Outdated Show resolved Hide resolved

invokeai/app/invocations/fields.py Outdated Show resolved Hide resolved

invokeai/app/invocations/flux_text_encoder.py Show resolved Hide resolved

invokeai/app/invocations/flux_text_encoder.py Outdated Show resolved Hide resolved

invokeai/app/invocations/flux_text_encoder.py Outdated Show resolved Hide resolved

invokeai/backend/quantization/load_flux_model_bnb_llm_int8.py Outdated Show resolved Hide resolved

invokeai/backend/requantize.py Outdated Show resolved Hide resolved

invokeai/backend/requantize.py Outdated Show resolved Hide resolved

invokeai/configs/flux/flux1-schnell.yaml Outdated Show resolved Hide resolved

pyproject.toml Outdated Show resolved Hide resolved

psychedelicious reviewed

View reviewed changes

invokeai/app/invocations/flux_text_encoder.py Outdated Show resolved Hide resolved

psychedelicious reviewed

View reviewed changes

invokeai/backend/model_manager/config.py Show resolved Hide resolved

psychedelicious reviewed

View reviewed changes

invokeai/backend/quantization/load_flux_model_bnb_llm_int8.py Show resolved Hide resolved

psychedelicious reviewed

View reviewed changes

invokeai/frontend/web/src/features/nodes/types/v2/common.ts Show resolved Hide resolved

psychedelicious reviewed

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

brandonrising force-pushed the brandon/flux-model-loading branch 2 times, most recently from 5606eea to 5edec7f Compare

August 21, 2024 13:11

RyanJDick force-pushed the brandon/flux-model-loading branch from ccb49c9 to e41025d Compare

August 21, 2024 18:21

RyanJDick reviewed

View reviewed changes

invokeai/backend/model_manager/load/model_loaders/flux.py Show resolved Hide resolved

psychedelicious reviewed

View reviewed changes

invokeai/app/invocations/flux_text_encoder.py Show resolved Hide resolved

StAlKeR7779 reviewed

View reviewed changes

invokeai/backend/flux/modules/autoencoder.py Outdated Show resolved Hide resolved

StAlKeR7779 reviewed

View reviewed changes

invokeai/backend/flux/modules/autoencoder.py Show resolved Hide resolved

RyanJDick reviewed

View reviewed changes

invokeai/app/invocations/flux_text_encoder.py Show resolved Hide resolved

RyanJDick reviewed

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

RyanJDick and others added 25 commits

August 26, 2024 19:21


          Update the T5 8-bit quantized starter model to use the BnB LLM.int8()…

54c48c3

… variant.


          Remove all references to optimum-quanto and downgrade diffusers.

5af214b


          Update docs for T5 quantization script.

f4612a9


          Move quantization scripts to a scripts/ subdir.

18c0ec3


          Downgrade revert torch version after removing optimum-qanto, and othe…

098db5c

…r minor version-related fixes.


          Update t5 encoder formats to accurately reflect the quantization stra…

dbdd851

…tegy and data type


          Switch the CLIP-L start model to use our hosted version - which is mu…

1d6c83b

…ch smaller.


          Replace swish() with torch.nn.functional.silu(h). They are functional…

1413ff9

…ly equivalent, but in my test VAE deconding was ~8% faster after the change.


          Setup scaffolding for in progress images and add ability to cancel th…

bd1b37d

…e flux node


          Remove dependency on flux config files

d159fe6


          ruff

877b88e


          Remove flux repo dependency

ae94e48


          Downgrade accelerate and huggingface-hub deps to original versions.

f046a38


          ruff format

9f6f404


          Remove outdated TODO.

9a530c7


          Only install starter models if not already installed

bb80697


          Remove in progress images until we're able to make the valuable

642a953


          Remove no longer used code in the flux denoise function

a90d098


          Fix type error in tsc

40a3fa5


          Run ruff

b9238b6


          Rename params for flux and flux vae, add comments explaining use of t…

5a5ca10

…he config_path in model config


          update default workflow for flux

bf59ab3


          remove prompt

bd2692b


          Run ruff

5d42e67


          default workflow: add steps to exposed fields, add more notes

c510234

brandonrising force-pushed the brandon/flux-model-loading branch from a30c2d9 to c510234 Compare

August 26, 2024 23:21

hipsterusername approved these changes

View reviewed changes


          Update starter model size estimates.

3b29bad

brandonrising merged commit 50085b4 into main

brandonrising deleted the brandon/flux-model-loading branch

August 27, 2024 00:17

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

psychedelicious psychedelicious left review comments

maryhipp maryhipp approved these changes

hipsterusername hipsterusername approved these changes

blessedcoolant Awaiting requested review from blessedcoolant blessedcoolant is a code owner

lstein Awaiting requested review from lstein lstein is a code owner

ebr Awaiting requested review from ebr ebr is a code owner

+2 more reviewers

StAlKeR7779 StAlKeR7779 left review comments

RyanJDick RyanJDick left review comments

Labels

backend CI-CD frontend invocations python python-deps python-tests Root services